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Formatting and Delivering Arbitrary Content 
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Field of the Invention 

The invention generally relates to viewing content with a wireless handheld 
device, and more particularly to viewing and interacting with content interactive 
10 contentthat is in a format incompatible with the handheld device, where an external 
system retrieves the incompatible content and converts it into compatible content. 

Background 



With the advent of fast and affordable networking equipment and Internet access, 



15 many new products have come to market that allow a consumer to augment everyday 
experiences with content obtained over a network. For example, for some television 
broadcasts, such as home shopping channels and game shows, the broadcaster may 
provide web sites containing content relevant to a particular show, item, or topic 
currently being broadcast to viewers. For example, QVC Inc. broadcasts items for 

20 display that may be purchased by a viewer having access to their Internet web site. 
Similarly, ABC's "Who Wants To Be A Millionaire" program simultaneously presents 
game show questions on an Internet web page so that viewers may play alongside 
contestants. 



25 Microsoft's "webtv" product, which incorporates an Internet browser within a television 
set top box. (Please note that all marks used herein are the property of their respective 



One attempt to provide web content along with a television broadcast is 
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owners.) Unfortunately, this integrated configuration, while convenient, presents 
several significant limitations. One limitation is that available screen real estate is 
limited if one combines display of television and interactive content. Alternatively, could 
elect whether to watch one or the other, but not both. Another limitation is that the 
5 content must be broadcast along with the viewing signal, e.g., over the cable television 
cable, which limits applicability of the product. Further, and more significantly, this 
solution requires usage of specialized hardware, e.g., inserters, encoders, decoders, 
processors, memory, etc., all of which significantly increases the cost of production for 
this solution. 

~~ 10 A second attempt to provide contemporaneous interactive content is to simulcast 

a? 

%,i interactive over the Internet. However, there are several significant problems with this 

01 solution. For example, this solution is not ergonomic, as it requires one to sit close to a 

fn 

03 personal computer display to receive the interactive content, and then look at a distance 

=P 

s to the television. This is due to a computer typically being located remote to the 

Uf> 15 television. Also, one's home computer typically is not even in the same room as the 

s i J 

n 1 

J=5 television, thus rendering this solution impractical as it requires one to leave the 
television viewing room to access the computer. Alternatively, one must install a 
computer system in the television viewing room, a solution many people would not like. 
A third attempt at contemporaneous interactive content to provide, such as by 
20 way of an Internet simulcast, interactive content to wireless handheld computers, such 
as Microsoft Windows CE machines or Palm Computing's Personal Information 
Managers (PIMs) running AvantGo, Inc.'s AvantGo software, or equivalent. These 
machines operate using a microbrowser that receives the provided interactive content, 
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and the handheld reformats received content for display on the handheld. Reformatting 
is necessary since general interactive content assumes a general computer system 
recipient, and not the limited capabilities of a handheld device. Unfortunately, ability to 
reformat content significantly increases the cost and complexity of the handheld device, 
5 as reformatting is nontrivial. 



Brief Description Of The Drawings 

The features and advantages of the present invention will become apparent from 
the following detailed description of the present invention in which: 
r * 10 FIG. 1 illustrates an exemplary hardware environment according to one 

sj embodiment of the invention. 

01 FIG. 2 illustrates an exemplary flow chart for retrieving interactive content for a 

Di 

CM handheld device while watching a television broadcast. 

3 

f FIG. 3 illustrates a suitable computing environment in which certain aspects of 

LJt 15 the invention may be implemented. 

ry 

M 

Detailed Description 

FIG. 1 illustrates an exemplary hardware environment according to one 
embodiment of the invention. 
20 A wireless handheld device 100 is in communication with a local computer 

system 102, such as a home personal computer (PC). The handheld device 
communicates with the local computer system 102 over a wireless channel 104, such 
as a radio frequency (RF), infrared (IR), or other wireless channel, using a 
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communication protocol such as (IEEE) 802.1 1 or other protocol. Various wireless 
transmission strategies may be used to convey data to the handheld device. For 
example, wireless communications may use direct or diffuse IR, narrowband RF 
transmissions, 900MHz or 2.4GHz transmissions, spread spectrum RF technology such 
5 as frequency hopping or direct sequence, etc. It is intended that any wireless 
communication channel and communication protocol may be used to implement 
wireless channel 104. 

The local computer system 102 acts as a computation engine to format incoming 
content, and it operates as a shared connection point to a network 106 and a storage 
n 10 device to cache additional content for the handheld device. In contrast with current 
si handheld devices that are required to reformat interactive content to conform the 

is 

01 content to the capabilities of the handheld device, in one embodiment, the power of the 

m 

*2 local computer system 102 is instead leveraged to perform this function. This allows the 
j\ construction of the handheld device to be simplified, thus reducing manufacturing costs 
j=fj 15 and risk of failure, e.g., less complex design leaves fewer elements to break. In the 
p claims that follow, the phrases "formatting agent" and "formatting server" refer to a local 
computer system and/or an application program that converts content for use by the 
handheld device. A formatting agent/server may also act as a storage agent or cache 
for content, and as a network access point for multiple handheld devices. 
20 In this embodiment, the handheld device is required only to support a single 

content format environment, e.g., a single environment or operating system for receiving 
user input and displaying output; in another embodiment, the handheld device supports 
a restricted number of content format environments. The local computer system 102 is 
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responsible for retrieving interactive content, which may be in any arbitrary format, such 
as Internet formats (e.g., HyperText Markup Language (HTML), Dynamic HTML 
(DHTML), Java, JavaScript, ActiveX), Standard formats (e.g., Extensible Markup 
Language (XML), Advanced Television Enhancement Forum (ATVEF), etc.), or a 
5 proprietary format (e.g. Sports tickers, etc.), and then converting the arbitrary format as 
required into the handheld device's supported environment. In one embodiment, the 
handheld local data format is the Wireless Application Protocol (WAP) Wireless Markup 
Language (WML) (a variant on XML designed for portable devices). 

It will be appreciated that some interactive content may be active or dynamic, 
q 10 e.g., making use of Java, Javascript, etc. In such cases, the local computer system 102 
v] executes the active or dynamic content and provides resultant data (e.g., output) to the 
01 handheld device. In effect, the local computer system acts as a simple input/output 
W device, where the handheld controls the local computer system. User input to the 

f s handheld is transmitted to the local computer system and used to control the active or 

hi 

|f i 15 dynamic content executing thereon, and results from such indirect user control of the 

ft i 

rj local computer system is converted and sent back to the handheld device. In one 

O 

embodiment, electronic mail (E-mail) services are provided to the handheld device. 

The local computer system 102 is communicatively coupled to a network 106, 
such as an intranet or Internet. In one embodiment, the local computer system retrieves 
20 networked resources from remote content providers 108, 110, 112. For example, such 
providers may offer games, news, shopping, etc., that are tied in or synchronized with 
television broadcasts, live sporting events, or other viewer experiences. It will be 
appreciated that any number of conventional wireless and wired techniques may be 
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used to connect the local computer system to the network 106, and that various 
networks may be used to communicate with content providers. As discussed above, 
the handheld device receives data from the local computer system 102 over wireless 
channel 104; however, note that the wireless channel may be carried, in part, over 
5 network 106. For example, a wireless base station (not illustrated) may be in 
communication with the local computer system 102 over the network 106. Thus, 
communication from the local computer system is sent over the network 106, to the 
base station, which in turn wirelessly communicates with the handheld device 100. 
The local computer system 102 is also communicatively coupled to a content 

10 initiator 114. Unlike the local computer system, the content initiator is likely physically 
remote from the local computer system and handheld device, and may also be 
communicatively coupled to the network 106. When a user of the handheld desires to 
engage in entertainment augmentation services, the content initiator is responsible for 
providing the local computer system with a catalog of services currently available to the 

15 user. The catalog of services corresponds to various interactive content that may be 
retrieved by the local computer system for conversion and provision to the handheld 
device. It will be appreciated that various known techniques can be used to ensure a 
local computer system is presented with a catalog that "makes sense" with respect to 
the location of the handheld device. For example, different geographic time zones are 

20 frequently provided with different television broadcasts, thus it may be necessary to 
ensure that the catalog is specific to a particular handheld device's location. 
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FIG. 2 illustrates an exemplary flow chart for retrieving interactive content for a 
handheld device while watching a television broadcast. 

A first operation is to watch 200 an entertainment broadcast, e.g., a television 
broadcast, such as a traditional terrestrial, satellite or cable broadcast, or a broadcast 
5 through a physical medium such as an intranet or Internet network. It will be 

appreciated that the illustrated flowchart applies to using the handheld device in other 
contexts besides augmenting television viewing. During watching, a user of the 
handheld device selects 202 a button, such as a physical button 116 (FIG. 1) within the 
casing of the handheld device, or a logical button 118 drawn on the display 120 of the 
P , 10 handheld device. This selection may occur irrespective of whether the user is watching 
d television. 

4= 

01 The illustrated embodiment assumes a television broadcast context, hence 

M selection 202 of the button indicates the user wants to engage in entertainment 
f augmentation, e.g., enhanced television making use of interactive content related to the 

JJ : ! 15 watched 200 broadcast. In one embodiment, the handheld device and television are 
pi completely decoupled, e.g., the television broadcast and providing of interactive content 
are entirely separate. Thus, the television may be an ordinary television receiving a 
signal from any source, such as an aerial antenna. In this embodiment, interactive 
content is simply provided contemporaneous to the television broadcast. The degree of 
20 synchronization between interactive content and the broadcast may be determined at 
the broadcast head end. In another embodiment, the content is embedded within the 
television signal, and extracted for conversion and presentation to the handheld device. 
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In response to selecting the button 116, 118, the handheld device, by way of the 
local computing device 102, transparently connects 204 to a content initiator 114. The 
content initiator may be operated by an entity that assembles various interactive content 
for delivery to requesting local computer systems 102. In one embodiment, there are 
5 multiple content initiators each providing different content arrangements to local 

computer systems for delivery to handheld devices. The handheld device requests 206 
a catalog of available services from the content initiator . The content initiator is 
responsible for assembling data to store in the catalog. This data may be retrieved by 
the content initiator through contacting content providers 108, 110, 112, and polling 

1 0 them for data, or by having catalog data pushed to the content initiator. 

In response to the request 206, the handheld device receives 208 the catalog 
over the wireless link 104. The catalog is displayed 210 on the handheld device display 
120, and a user selects 212 a desired content category, e.g., Sports broadcast, 
Electronic Program Guide (EPG), entertainment TV shows. In one embodiment, catalog 

15 content categories comprise links to content provider 108, 1 10, 112 content (e.g., 
television guides, interactive entertainment, order processing systems, etc.), and 
selection 212 of a category directs the local computer system 102 to contact 214 the 
appropriate linked content provider. 

In response to contacting a linked content provider 108, 110, 112, the local 

20 computer system receives 216 content "feed" from the linked content provider in a first 
data format, e.g., HTML, DHTML, XML, Java code, etc. In one embodiment, the local 
computer system tests 218 whether this first data format is compatible with the 
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handheld device, e.g., can the content be directly forwarded to the handheld device for 
appropriate processing. 

If the first data format is incompatible with the handheld device, the processing 
power of the local computer system 102 is utilized to convert 220 the first data format 
5 into a second data format that is compatible with the handheld device, e.g., the local 
computer system acts as a formatting agent for the handheld device. For example, in 
one embodiment, the handheld device is configured to process WML encoded data. 
When the first data format is non-WML interactive content, the local computer system 
converts it into the compatible second data format WML content. 

10 In another embodiment, conversion is based on handheld device characteristics. 

Thus, if the handheld device has limited memory, content images may be deleted, 
reduced in size, replaced with placeholders, or streamed to the handheld for scrolling 
across the screen as necessary. (For scrolling an image past the display, no storage is 
required beyond display needs since the image is not stored on the handheld device.) If 

15 the handheld device has a non-color display, color data may be removed from content. 
If the content includes forms or other web page data, these forms and other data can be 
converted and re-formatted, if necessary, to best-fit the handheld device display. 

In another embodiment, the storage capacity of the local computer system 102 is 
used to cache content received from the network. This cached data may be displayed 

20 on the handheld device. An exemplary use for cached data is caching a weekly 
download of an Electronic Program Guide (EPG). The EPG is likely to be accessed 
frequently, and caching the guide allows it to be displayed quickly, and also allows it to 
be displayed without use of a network connection. 
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Once the first data format is converted 220 into the second data format, it is sent 
222 to the handheld device for processing. If 218 the content was originally compatible, 
as may be found, for example, at select sites directly supporting the handheld device, 
then the original content is sent 224 to the handheld device for processing. 

5 

FIG. 3 and the following discussion are intended to provide a brief, general 
description of a suitable computing environment in which certain aspects of the 
illustrated invention may be implemented. 

An exemplary system for implementing the invention includes a computing 

1 0 device 300 having system bus 302 for coupling various computing device components. 
This system, or portions thereof, may be used to implement the FIG. 1 handheld device 
100, local computer system 102, content providers 108-112, and content initiator 114. 
Typically, attached to the bus are non-programmable and programmable processors 
304, a memory 306 (e.g., RAM, ROM), storage devices 308, a video interface 310, and 

15 input/output interface ports 312. Storage devices include hard-drives, floppy-disks, 
optical storage, magnetic cassettes, tapes, flash memory cards, memory sticks, digital 
video disks, and the like. 

Portions of the invention may be described by reference to different high-level 
program modules and/or low-level hardware contexts. Those skilled in the art will 

20 realize that program modules can be interchanged with low-level hardware instructions. 
Program modules include procedures, functions, programs, components, data 
structures, and the like, for performing particular tasks or implementing particular 
abstract data types. Modules may be incorporated into single and multi-processor 
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computing devices, such as the local computer system, content providers, content 
initiators, handheld devices such as the device illustrated in FIG. 1, Personal Digital 
Assistants (PDAs), cellular telephones, and the like. Thus, the storage systems and 
associated media can store data and executable instructions for the computing device. 
5 Modules may be implemented within a single computing device, or processed in a 
distributed network environment, and stored in both local and/or remote memory. 

The computing device is expected to operate in a networked environment using 
logical connections to one or more remote computing devices 314, 316 through a wired 
or wireless network interface 318, modem 320, or other communication pathway. 
C1 10 Computing devices may be interconnected by way of a network 322 such as an intranet, 
Q the Internet, or other network. This network may be, in whole or in part, the FIG. 1 
01 network 106. Thus, for example, with respect to the illustrated embodiments, assuming 
M computing device 300 is the handheld device 100, then by way of the local computer 
■ , system 102, remote devices 314, 316 may respectively be the content initiator 114 and 

as ; 

LH 15 a content provider 108, 110, 112. The FIG. 1 handheld device 100, by way of the local 

p computer system 102, may receive content catalogs and content provider 108-1 12 

pi 

content distributed over the network 322. 

It will be appreciated that remote computing devices 314, 316 may be configured 
like computing device 300, and therefore include many or all of the elements discussed 
20 for computing device. It should also be appreciated that computing devices may be 
embodied within a single device, or separate communicatively-coupled components, 
and may include or be embodied within routers, bridges, peer devices, web servers, and 
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application programs utilizing network application protocols such as the HyperText 
Transfer Protocol (HTTP), File Transfer Protocol (FTP), and the like. 

Having described and illustrated the principles of the invention with reference to 
5 illustrated embodiments, it will be recognized that the illustrated embodiments can be 
modified in arrangement and detail without departing from such principles. 

And, even though the foregoing discussion has focused on particular 
embodiments, it is understood that other configurations are contemplated. In particular, 
even though expressions such as "in one embodiment," "in another embodiment," or the 
10 like are used herein, these phrases are meant to generally reference embodiment 
possibilities, and are not intended to limit the invention to particular embodiment 
configurations. As used herein, these terms may reference the same or different 
embodiments, and unless implicitly or expressly indicated otherwise, embodiments are 
combinable into other embodiments. Consequently, in view of the wide variety of 
15 permutations to the above-described embodiments, the detailed description is intended 
to be illustrative only, and should not be taken as limiting the scope of the invention. 

What is claimed as the invention, therefore, is all such modifications as may 
come within the scope and spirit of the following claims and equivalents thereto. 

19 
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